airflow源码精读二

您所在的位置：网站首页 › airflow webserver › airflow源码精读二

airflow源码精读二

#airflow源码精读二| 来源: 网络整理| 查看: 265

Building the image 构建镜像

Installing from Docker Images

https://airflow.apache.org/docs/docker-stack/index.html

# apt安装包 FROM apache/airflow:2.5.1 USER root RUN apt-get update \ && apt-get install -y --no-install-recommends \ vim \ && apt-get autoremove -yqq --purge \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* USER airflow # pypi 安装包 FROM apache/airflow:2.5.1 RUN pip install --no-cache-dir lxml # 用requirements.txt安装包 FROM apache/airflow:2.5.1 COPY requirements.txt / RUN pip install --no-cache-dir -r /requirements.txt # Embedding DAGs 嵌入加载dag FROM apache/airflow:2.5.1 # 把test_dag.py以用户airflow 用户组root copy到镜像/opt/airflow/dags中 COPY --chown=airflow:root test_dag.py /opt/airflow/dags

test_dag.py如下

import datetime import pendulum from airflow.models.dag import DAG from airflow.operators.empty import EmptyOperator now = pendulum.now(tz="UTC") now_to_the_hour = (now - datetime.timedelta(0, 0, 0, 0, 0, 3)).replace(minute=0, second=0, microsecond=0) START_DATE = now_to_the_hour DAG_NAME = "test_dag_v1" dag = DAG( DAG_NAME, schedule="*/10 * * * *", default_args={"depends_on_past": True}, start_date=pendulum.datetime(2021, 1, 1, tz="UTC"), catchup=False, ) run_this_1 = EmptyOperator(task_id="run_this_1", dag=dag) run_this_2 = EmptyOperator(task_id="run_this_2", dag=dag) run_this_2.set_upstream(run_this_1) run_this_3 = EmptyOperator(task_id="run_this_3", dag=dag) run_this_3.set_upstream(run_this_2)

启用Buildkit来构建映像

DOCKER_BUILDKIT=1 Using docker-context-files

build镜像时需要添加参数--build-arg DOCKER_CONTEXT_FILES=docker-context-files

mkdir -p docker-context-files cat

【本文地址】

airflow源码精读二

airflow源码精读二

今日新闻

推荐新闻

airflow源码精读 二

airflow源码精读 二

今日新闻

推荐新闻

airflow源码精读二

airflow源码精读二